Sharing encryption-related metadata between a host and an external intermediate device

ABSTRACT

A method performed in a device is provided. The method includes (a) receiving key identification information from a key controller module on an external host, (b) obtaining a key identified by the key identification information from an external key server, (c) decrypting encrypted data from an encrypted storage system using the key, and (d) processing the decrypted data. A corresponding computer program product is also provided.

CROSS REFERENCE TO RELATED APPLICATION

This Patent Application is a Divisional of U.S. patent application Ser.No. 12/977,789 filed on Dec. 23, 2010, entitled, “SHARINGENCRYPTION-RELATED METADATA BETWEEN MULTIPLE LAYERS IN A STORAGE I/OSTACK.” The contents and teachings of the above-identified PatentApplications are hereby incorporated by reference in their entirety.

BACKGROUND

In computer systems, it is sometimes desirable to encrypt some or all ofthe data stored in storage devices in the system. In some arrangements,data is encrypted in an I/O filter driver running on a host of thecomputer system. In some configurations, the I/O filter driver isconfigured to use a key securely provided by a network key server sothat multiple hosts within a security domain can securely access thesame data.

SUMMARY

The above-described approach to encryption in computer systems may notbe entirely optimal, because encryption is a cycle-intensive task, soperforming the encryption in a software filter driver on the host cancause slow performance. It may be desirable to offload data encryptiontasks to specialized hardware devices which operate under the control ofsupervisory software components. In other systems, there may be othersystem components that perform data encryption tasks under suchsupervision. In these kinds of systems it is necessary for such dataencrypting components to obtain encryption “metadata”, such as dataencryption keys, for use in the data encryption operations.

Embodiments of the present invention are directed to techniques forsharing encryption-related metadata between layers of a storage I/Ostack so that the metadata can be effectively transferred thereby tospecialized hardware devices or other encrypting components in acomputer system. The disclosed techniques can provide for efficient andsecure passing of encryption information among system elements to enablea variety of system functions.

A method performed in a device is provided. The method includes (a)receiving key identification information from a key controller module onan external host, (b) obtaining a key identified by the keyidentification information from an external key server, (c) decryptingencrypted data from an encrypted storage system using the key, and (d)processing the decrypted data. A corresponding computer program productis also provided.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features and advantages will beapparent from the following description of particular embodiments of theinvention, as illustrated in the accompanying drawings in which likereference characters refer to the same parts throughout the differentviews. The drawings are not necessarily to scale, emphasis instead beingplaced upon illustrating the principles of various embodiments of theinvention.

FIG. 1 depicts an example system for use in practicing variousembodiments of the invention.

FIG. 2 depicts an example apparatus for use in practicing variousembodiments of the invention.

FIG. 3 depicts an example view of various components for use inpracticing various embodiments of the invention.

FIG. 4 depicts an example logical layout of a system for use inpracticing various embodiments of the invention.

FIG. 5A depicts an example scenario according to one embodiment of theinvention.

FIG. 5B depicts an example scenario according to another embodiment ofthe invention.

FIG. 6 depicts an example method according to one embodiment of theinvention.

FIG. 7 depicts an example Associate Command Block according to oneembodiment of the invention.

FIG. 8 depicts an example method according to one embodiment of theinvention.

DETAILED DESCRIPTION

FIG. 1 depicts an example distributed computer system 30 (capable ofperforming as an encrypted data storage system) for use in performingvarious embodiments of the invention. System 30 includes a key managerserver 32, a host interconnect 34, and one or more hosts 36 (depicted ashosts 36(a), 36(b), . . . , 36(n)). Key manager server 32 and hosts 36connect to each other via host interconnect 34. The hosts 36 alsoconnect to storage devices 40 (depicted as storage devices 40(a), 40(b),. . . , 40(m)) via a storage interconnect 38. In some embodiments, thehost interconnect 34 and the storage interconnect 38 are combined. Oneor more external intermediate devices 42 may also attach to hostinterconnect 34 and storage interconnect 38. Certain types of externalintermediate devices 42 may work in conjunction with a WAN tunnel 44 andone or more remote storage devices 46, as described further below.

In operation, the hosts 36 execute application programs that utilize thestorage devices 40 for non-volatile data storage. The storageinterconnect 38 may employ a storage-oriented protocol such as iSCSI orFibre Channel to enable block-oriented read and write commands and theaccompanying data to be transferred between the hosts 36 and storagedevices 40. Additionally, the system 30 provides selective encryption ofstorage data by the hosts 36 (and potentially the external intermediatedevice 42). The key manager server 32 and host interconnect 34 providesupport for the data encryption function as described in more detailbelow.

Key manager server 32 provides key manager functionality, i.e., thegeneration, protection, storage, replacement, and elimination of dataencryption keys and related data that are used in dataencryption/decryption operations. In one embodiment, key manager server32 is a server appliance. One example of a key manager server 32 usablein some embodiments is the RSA Key Manager appliance manufactured by EMCCorp. of Hopkinton, Mass. It should be understood that this is by way ofexample only; other products may also serve as the key manager server32.

Key manager server 32 and hosts 36 connect to each other via hostinterconnect 34. Host interconnect 34 may be, for example, a network,such as a local area network (LAN) or a wide area network (WAN). Hostinterconnect 34 may also be realized by a collection of one or moreswitches interconnecting key manager server 32 and hosts 36.

Hosts 36 are computers executing applications that store data on thedata storage devices 40. In addition to connecting to the hostinterconnect 34, each host 36 also connects to the storage interconnect38, typically via a plurality of independent connections. In oneembodiment, the hosts 36 employ a multipathing function whichestablishes and utilizes multiple paths from a given host 36 to a givenstorage device 40, which can provide higher performance as well asredundancy for greater availability. Further detail regarding hosts 36is provided below in connection with FIGS. 2 and 3.

The storage interconnect 38 can be any type of network or input/output(I/O) bus capable of interconnecting storage devices 40 with hostcomputers 36. In some embodiments, the storage devices 40 and host 36are interconnected in a manner such that, to the operating systemsrunning on the hosts 36, the storage devices 40 appear as locallyattached, but this is not required for the invention. The storageinterconnect 38 may be a shared, public, or private network andencompasses a wide area or local area and can be implemented through anysuitable combination of wired and/or wireless communication networks.Furthermore, the storage interconnect 38 may include a LAN, a WAN, anintranet, the Internet, or a set of switches. For example, in oneembodiment, the storage interconnect 38 works with Fibre Channelconnectivity and is implemented in the form of a storage area network(SAN). In another embodiment, the storage interconnect 38 works withinternet protocol (IP) connectivity and is implemented via anInternet-Small Computer System Interface (iSCSI) (e.g., for FibreChannel). Those of skill in the art will recognize that otherimplementations are, of course, possible.

Storage devices 40 may be any sort of storage equipment capable ofconnecting to storage interconnect 38. In some embodiments, each storagedevice 40 is a disk array. As is well-known in the art, a typical diskarray includes a disk array controller, disk enclosures holding aplurality of disk drives, and a power supply. A disk array may alsoinclude a cache. Examples of disk arrays include the SymmetrixIntegrated Cache Disk Array System and the CLARiiON Disk Array System,both available from EMC Corp. of Hopkinton, Mass.

In some embodiments, an external intermediate device 42 also attaches tohost interconnect 34 and storage interconnect 38, for example, toprovide a data de-duplication feature, or, in connection with WAN tunnel44 and remote storage device 46, a remote replication feature.Additional detail is provided below.

As mentioned, key manager server 32 controls the generation, protection,storage, replacement, and elimination of data encryption keys. Inparticular, key manager server 32 creates encryption keys andcorresponding key identifiers. Each key identifier, referred to as akey_id, is associated with a corresponding encryption key and can beused to obtain the key from the key manager server 32, provided that allpermissions and credentials are in place.

FIG. 2 depicts a host 36 in greater detail. Each host 36 includes a hostinterface 50 for connecting to host interconnect 34, a processor 52,memory 54, and one or more host bus adapters (HBA) 56 (depicted as HBAs56(a), 56(b), . . . , 56(p)) for connecting to storage interconnect 38over redundant paths. Processor 52 may be any sort of controller, suchas, for example, a general purpose processor or microprocessor, acentral processing unit, a set of multiple processing units, or a set ofdedicated circuitry designed to perform particular operations inhardware. Memory 54 may be made up of one or more of the following:volatile random access memory, non-volatile read-only memory,non-volatile flash memory, magnetic storage, optical storage, etc. Insome embodiments, one or more of the HBAs 56 are “encrypting” HBAs thatperform encryption and decryption of storage data using dedicatedhardware circuitry which is not shown in FIG. 2.

FIG. 3 illustrates certain software that is contained within the memory54 during system operation. As shown, in one embodiment, memory 54stores one or more computer program applications 58 and an operatingsystem (OS) 60. Applications 58 and OS 60 contain a set of instructionsto be executed by processor 52. Memory 54 may also store applicationdata.

OS 60 (which contains many well-known components that are not shown ordescribed herein) includes a file system 62 and a logical volume manager64. OS 60 also includes an input/output (I/O) filter driver 65 and anHBA driver 67. I/O filter driver 65 may be, for example, a component ofthe PowerPath Encryption With RSA software available from EMC Corp. ofHopkinton, Mass. I/O filter driver 65 includes an OS interface 68, anHBA interface 70, and a set of common application programming interfaces(APIs) 72. I/O filter driver 65 also includes a key controller module(KCM) or encryption manager 74 and one or more intermediate layers (IL)76. ILs 76 may include, for example, one or more virtualization modules80 and multipathing modules 82. Crypto kernel 84 may also be consideredto be part of I/O filter driver 65. Portions of the I/O filter driver 65and the HBA driver 67 may also make up storage I/O stack 66. It shouldbe understood that this arrangement is by way of example only; in someembodiments, one or more components of the storage I/O stack 66 may beexternal to the I/O filter driver 65. In any case, for purposes of thisDisclosure, the storage I/O stack 66 includes components between the KCM74 and a software interface to the encryption endpoint (EE) whereencryption is performed (e.g., HBA driver 67, or in some cases, a driverfor external intermediate device 42).

The KCM 74 is generally responsible for managing the data encryptionaspects of operation of the host 36 in which it resides. In somearrangements, the KCM 74 may arrange for the encryption to be performedby crypto kernel 84. However, since KCM 74 and crypto kernel 84 both runin software (running on processor 52), such operation may impose aperformance penalty in terms of latency and/or throughput of datastorage operations. Therefore, in some arrangements, KCM 74 is able toarrange for the encryption to be performed by a hardware encryptingcircuit, referred to as a “hardware assist,” which may be located withinone or more HBAs 56 as mentioned above. An HBA 56 that includes ahardware assist may be referred to as an encrypting HBA or “EHBA”, whilean HBA 56 that does not include a hardware assist may be referred to asa non-encrypting HBA or “NHBA”.

FIG. 4 depicts an example logical arrangement 86 of storage I/O stack 66and other system elements according to one embodiment. In particular,FIG. 4 depicts functional connections within the storage I/O stack 66and between the storage I/O stack 66 and certain disks of the storagedevices 40 via respective HBAs 56. The disks 140(1)-140(4) are labeledD1-D4 respectively. The HBAs 56 are shown as EHBAs 156(1)-156(3) and anNHBA 256(1).

A logical disk L1 88(a) is defined by virtualization module 80.Virtualization module 80 provides a “virtualization” system function,presenting a logical unit of data (LU) as a logical disk or logicalvolume (LV) to KCM 64 and to the OS 60 via OS interface 68 even thoughthe LV may not actually be a contiguous physical entity, which isassumed to result in assigning logical blocks of L1 to specific storagedevices 40. This virtualization may be, for example, a mirroring, astriping, or some combination thereof. In arrangement 86, logical diskL1 88(a) is shown as being virtualized across two storage devices D1140(1) and D4 140(4). It should be understood that, throughout thisDescription, the term LU is used to refer to a logical unit of data atany level of abstraction (e.g., as seen by the KCM 74, as seen by one ofthe ILs 76, or as seen by an HBA 56), while the term LV is used tospecifically refer to an LU as seen by the KCM 74.

A multipathing module 82 provides a multipathing system function bywhich multiple paths to these storage devices are established throughthe storage interconnect 38 and utilized in operation for greaterparallelism, availability, and performance. As depicted, multipathingmodule 82 connects to EHBA1 156(1), EHBA2 156(2), EHBA3 156(3), andNHBA1 256(1) (via the HBA driver interface 70 and HBA driver 67 of FIG.3), and the following paths exist:

To D1 140(1) via EHBA1 156(1), EHBA3 156(3), and NHBA1 256(1)

To D2 140(2) via EHBA2 156(2) and EHBA3 156(3)

To D3 140(3) via NHBA1 256(1)

To D4 140(4) via EHBA1 156(1) and NHBA1 256(1).

It should be noted that FIG. 4 presents a simplified example whichassumes that each HBA 56 and storage device 140 has only one connectionto the storage interconnect 38. In general, as depicted in FIG. 1, eachHBA 56 and storage device 140 may have multiple such connections, and itwill be appreciated that the number of potential paths between a givenHBA 56 and storage device 140 may be correspondingly greater.

In the configuration of FIG. 4, the only path to disk D3 140(3) is viaNHBA1 256(1), which means that there is no hardware assisted encryptionavailable for encrypting/decrypting data of that disk. The significanceof this incapability is described below.

In an arrangement such as that of FIG. 4, the multipathing module 82 isresponsible for maintaining an awareness of which disks 140 it can“reach” (engage in I/O operations with) as well as the corresponding setof usable paths to each reachable disk. The virtualization module 80maintains an awareness of the disks (e.g., D1 140(1) and D4 140(4))which underlie each logical volume (e.g., L1 88(a)). Upon receivingstorage commands (I/O commands including reads and writes of storagedata) directed to logical volume L1 88(a), the virtualization module 80generates corresponding storage commands to D1 and D4 and issues thesecommands to the multipathing module 82. The multipathing module 82responds by selecting a path for each command and issuing the command tothe HBA 56 for the selected path. Storage commands directed to anencrypted region of a disk 140 may utilize the hardware assist providedby an EHBA 156 along a selected path. In the event that a disk 140 isnot reachable via an EHBA 156 (such as disk D3 140(3) as mentionedabove), any such storage commands will utilize the encryptionfunctionality of the crypto kernel 84.

FIGS. 5A and 5B illustrate specific examples of the above-describedoperation in greater detail. FIG. 5A depicts one arrangement 90 fordisks D1 and D4 of logical volume L1. The diagram shows that disk L188(a) may be encrypted using hardware assist, because each underlyingstorage device D1, D4 that L1 maps to can be accessed through an EHBA156. Thus, since storage device D1 may be accessed via EHBA1 and EHBA3(indicated by the “Yes” along the connections between multipathingmodule 80 and 156(1) and 156(3)), and D4 may be accessed through EHBA1(indicated by the “Yes” along the connection between multipathing module82 and 156(1)), a combination of EHBA1 and EHBA3 may be used to performall encryption operations for accessing logical disk L1 88(a). It shouldbe noted that although both storage devices D1 and D4 are accessiblethrough NHBA1 256(1), a “No” is depicted along the connections betweenNHBA1 256(1) and multipathing module 82 because NHBA1 256(1) does notprovide a hardware assist feature.

FIG. 5B depicts an alternate arrangement 92 for a second logical disk L288(b) and its underlying disks D2 140(2) and 140(3). As shown, disk D3is accessed only via NHBA1 256(1). Therefore, encrypted storageoperations for logical disk L2 88(b) do not utilize hardware assist,because not all of its component storage devices can be accessed by anEHBA 156. Thus, in arrangement 92, crypto kernel 84 is used to performthe encryption operations required by any data storage operations to beperformed on logical disk L2 88(b).

The above description in connection with FIGS. 4 and 5A-5B illustratescertain important aspects of using hardware assisted encryption in asystem such as that of FIG. 1. First, it must be possible for an EHBA156 to obtain the encryption metadata (including encryption key) forthose regions of encrypted storage for which that EHBA 156 will handledata storage commands, so that the hardware encryption circuitry of theEHBA 156 can perform the correct encryption/decryption operation usingthe correct key for each distinct region of storage. As the KCM 74 isthe overall manager of encryption operations for the host 36 in which anEHBA 156 resides, a mechanism is needed to enable the KCM 74 tocommunicate the encryption metadata to its EHBAs 156. Additionally, amechanism is needed for the KCM 74 to ascertain whether hardwareassisted encryption is available for any given region of storage. Boththese needs are further complicated by the presence of ILs 76,especially those (like virtualization module 80) which are “remapping”layers that effect a translation or mapping between two differentrepresentations of a given storage volume. Additionally, evennon-remapping layers like the multipathing module 82 create potentialproblems, because hardware encryption assist may not be available on allpaths for a given disk 140, yet the system must ensure that encryptionis performed reliably. All these issues point to the need for acommunications protocol among the different layers of the storage I/Ostack 66 to support the data encryption function.

FIG. 6 depicts an example method 1000 for setting up encryption on alogical disk (e.g., logical disk L1 88(a)) and then performing encryptedstorage operations in an efficient manner. FIG. 6 is directed to thespecific case of hardware-assisted encryption, but several aspects ofthe method are more generally applicable to the use of other types ofEEs, which perform cryptographic processing (e.g., encryption,decryption, or both), as explained below. In one embodiment the methodis performed using an “in-band” communications protocol among thevarious components of the storage I/O stack 66. Here “in-band” refers tothe fact that communication related to the method is performed along thesame path as the I/O. In one embodiment, specialized SCSI commands andresponses, for example, are transported up and down the storage I/Ostack 66 using the same transport mechanism which is used to convey theSCSI commands that contain storage commands (reads and writes) andresponses. In some embodiments, special commands are embedded in theSCSI buffer on a SCSI read command. This communications protocol isreferred to below as a “DEK management protocol,” where the acronym DEKstands for “data encryption key.”

By “up and down” the storage I/O stack 66 it is meant that a DEKmanagement protocol command may be created by KCM 74 then passed to atop-level IL 76, such as virtualization module 80. That IL 76 examinesthe command and, in most cases (exceptional cases are discussed below),will send one or more corresponding commands to the next IL 76 down thestack, such as multipathing module 82. This pattern repeats until one ormore commands reach HBA driver(s) 67. Responses flow in the otherdirection, from the HBA drivers 67 upward to the KCM 74. In some cases,commands may not travel completely down the storage I/O stack 66, andresponses may be generated and sent upwards by one or more ILs 76. Bythis chain-like communications mechanism, information required forproper encryption-related operation is shared among the variouscomponents of the storage I/O stack 66.

In one embodiment, KCM 74 uses the DEK management protocol to firstdetermine whether or not there is an EHBA 156 (or a set of multipleEHBAs 156) that can provide hardware encryption for each encryptedregion of the logical disk L1 88(a). If not, then it is deemed thathardware encryption is not available, and the KCM 74 assumesresponsibility for encryption/decryption operations for the logical diskL1 using the crypto kernel 84. If the KCM 74 determines that suchhardware encryption is available, it uses the DEK management protocol toprovide the required encryption metadata to each EHBA 156 that requiresit. Subsequently, storage commands directed to the logical disk L1 aresent down the stack 66 for execution, relying on operation of one ormore EHBAs 156 for the data encryption/decryption part of operation forthe encrypted regions.

As previously noted, encryption may be applied to separate “regions” ofa given volume 88 or disk 140. Here “region” refers to a span ofcontiguous logical block addresses (LBAs). To illustrate the concept,assume a hypothetical simple volume 88 having 16 blocks of storage withaddresses 0 through 15. The volume may have an encryption patterns asfollows:

LBA range Encryption? 0-3 Not encrypted  4-12 Encrypted 13-15 Notencrypted

The overall pattern for a given logical unit of data (LU) is referred tobelow as a “LUN map” (the term “LUN” is commonly used in the industry torefer to an LU). In operation, it is necessary for the KCM 74 to providethe LUN map for each volume to any EHBA 156 that will handle I/O forthat volume. It is assumed herein that only one data encryption key isused for each volume, although in general it is possible to usedifferent keys for different regions, for example.

In a somewhat more realistic example, an encrypted LU may store metadataand formatting information in plaintext form. In addition, certainadditional regions of an encrypted LU may be designated as unencryptedfor various reasons (e.g., to enhance performance on a region that isfrequently accessed). For example, logical disk L1 88(a) may be anencrypted LU having a size of 10 megabytes. Given a 512-byte block size,logical disk L1 88(a) has 20,480 blocks. Blocks 0-1023 may beunencrypted and reserved for operating system use, while blocks1024-1535 may be unencrypted and reserved for storing encryptionmetadata. Blocks 1536-10,239 may be encrypted, blocks 10,240-11,263 maybe unencrypted for performance reasons, and blocks 11,264-20,479encrypted. Thus, only blocks 1536-10,239 and 11,264-20,479 of logicaldisk L1 88(a) are subject to encryption.

Additionally, the virtualization module 80 distributes the blocks oflogical disk L1 88(a) out across D1 140(1) and D4 140(4). For example,blocks 0-10,239 may be stored on D1 140(1), while blocks 10,240-20,479are stored on D4 140(4). This arrangement places portions of logicaldisk L1 88(a) subject to encryption on both D1 140(1) and D4 140(4). Itshould be noted that the mapping between L1 and D1/D2 may not (and inmany cases will not) preserve LBAs. Thus blocks 0-10,239 of L1 may bemapped to blocks 32,000-42,239 of D1, for example.

Referring again to FIG. 6, in step 1010, KCM 74 determines if thereexist one or more EEs (e.g., EHBA(s) 156) that can performencryption/decryption for all encrypted regions of an encrypted logicalvolume (LV). If step 1010 returns an affirmative response, executionproceeds with step 1050, while if step 1010 returns a negative response,execution may proceed with step 1070. At step 1070, it is concluded thatthere is no EE to perform encryption/decryption for the subjectvolume/disk, which means that any required encryption/decryptionoperations are to be performed by the KCM 74 using the crypto kernel 84.As described above with reference to FIGS. 5A and 5B, in the presentexample the condition of step 1010 is satisfied for logical disk L188(a). However, for logical disk L2 88(b), step 1010 evaluates in thenegative because there is no encrypted path to D3 140(3), and thus themethod will execute step 1070 with respect to logical disk L2 88(b). Itshould be noted that in some embodiments, even if an EHBA 156 is presentin all paths to a storage device 40, condition 1010 could still fail ifan essential path (or an essential group of paths) is blocked by anerror in the EHBA 156 (e.g., the EHBA 156 has no remaining capacity oris temporarily offline).

As shown in FIG. 6, step 1010 may be accomplished by performing some orall of sub-steps 1012 and 1016, which perform handshake and queryoperations. In connection with these sub-steps, different specificcommands and responses of the DEK management protocol are used asdescribed more fully below. Table 1 provides a general structure for aDEK management protocol command block used in performing theseoperations:

TABLE 1 General command block format Bytes Field 0-7 (8 bytes) ProtocolSignature  8-15 (8 bytes) Checksum 16-19 (4 bytes) Version 20-23 (4bytes) Reserved 24-27 (4 bytes) DEK Management protocol Opcode 28-31 (4bytes) DEK Management protocol Response Status 32-39 (8 bytes) DEKManagement protocol Endpoint ID 40-47 (8 bytes) Key Controller Handle 48-1535 Command Specific Parameters and Data

The general command block format is a structure having a format as shownin Table 1. In some embodiments, all commands are a maximum of 1536bytes (3×512-byte blocks) long, although this is by way of example only.In some embodiments, DEK management protocol command blocks areimplemented within the read buffer of SCSI Read commands. The fields andarguments are described below. In the description below, the label [In]means the parameter is an “input” passed from the KCM 74 in/down to theEE (via one or more ILs 76), while [Out] means the parameter is an“output” returned by the EE out/up to the KCM 74 (via one or more ILs76). “Initiator” means the KCM 74 or cooperating IL 76 that generates aprotocol command. “Device Object” means a device managed by a driver inthe storage I/O stack 66. It may be a volume, an LV, an LU, a pathdevice, or a storage device.

The following is a description of the various fields in the generalcommand block shown in Table 1 above:

-   -   Protocol Signature—8 bytes—[In] identifies the contents as a DEK        Management protocol Command, to distinguish DEK management        protocol communications from other communications using the same        in-band transport. During the “Handshake” command of step 1012        (described below), the signature is set to a predefined value        (e.g., 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11) by the        KCM 74. The signature is echoed back unchanged by the EE for all        commands.    -   Checksum—8 bytes—[In] Used as a means of validating a DEK        management protocol command block. Contains a 32 bit        cyclic-redundancy-check checksum of bytes 16 to 1535, stored in        a longword. Calculated by the KCM 74 before sending the command        down.    -   Version—4 bytes—[In] DEK Management protocol version identifier.    -   DEK management protocol Opcode—4 bytes—[In] DEK Management        protocol operation to be performed. Opcodes includes values for        Handshake, Query, Associate, Update, and Disassociate. If an EE        had been doing encryption for a Device Object, it can release        any resources associated with that object after a Disassociate        command and can keep them released until it sees a new Associate        command. Associations are described below.    -   DEK management protocol Response Status—4 bytes—[Out] Reports        completion status for the protocol command. Set by the EE.        Examined by the ILs 76 and potentially modified by them.        Examined by the KCM 74. Valid values for this field can include        Success as well as various values representing failure due to a        number of possible errors.    -   DEK management protocol Endpoint ID—8 bytes—[In/Out] Unique        identifier for the EE's use. Its content varies by Opcode:        returned up to the KCM 74 on a Handshake and echoed (sent back        down to) the EE in Query, Associate, Update, and Disassociate        commands.    -   Key Controller Handle—8 bytes—[In] Contains a handle used by the        Initiator to match a DEK management protocol response with its        originating command. The EE and ILs 76 should echo/return this        field unchanged.    -   Command Specific Parameters and data—Up to 1488 bytes of        command-specific data. Examples are discussed below.

Referring again to FIG. 6, in step 1012 (which may be omitted), KCM 74sends a Handshake command down to the next IL 76 in order to test forthe existence of a cooperating EE in the storage I/O stack 66.

The KCM 74 sends the Handshake command as the first DEK managementprotocol command to a Device Object. Only one handshake at a time shouldbe outstanding to a given device object. The EE should not trigger aninternal state change upon receipt of a Handshake, e.g., a Handshakecommand should not reset associations currently in effect for a DeviceObject.

Generally, KCM 74 will send one Handshake command per LV that itmanages. As long as KCM 74 receives one affirmative Handshake responsecommand block in response to the Handshake command, KCM 74 will proceedto step 1016. Otherwise, KCM 74 will proceed with software encryption.

When a cooperating IL 76 receives a Handshake command from above in thestorage I/O stack 66, it passes the command down to the next level downin the storage I/O stack 66. If the LU potentially involves multiple EEsof the storage I/O stack 66, then the IL will replicate the commandblock (potentially with modification as discussed below) and send a copydown the storage I/O stack 66 toward each such EE.

For example, if the IL 76 is a virtualization module 80 that virtualizesthe LU across two storage devices 40(a) and 40(b), then virtualizationmodule 80 sends two copies of the Handshake command block down thestorage I/O stack 66, one referencing storage device 40(a) as the deviceobject, and the other referencing storage device 40(b) as the deviceobject. If the virtualization module 80 receives any affirmativeHandshake response command blocks, the virtualization module 80 respondsback to the KCM 74 with an affirmative response. This indicates to theKCM that there is at least one EE that may require encryption metadata.However, it should be understood that in some embodiments, some ILs 76may be configured to respond negatively if any of the Handshake responsecommand blocks from below are negative.

The DEK management protocol supports multiple “classes” of EEs. An EE ofa cooperating class ignores Handshake commands that do not contain itsclass name and acknowledges a Handshake addressed to its EE class nameby filling in the Endpoint ID field.

See Table 2, below, for an example layout of a Handshake command blockwith Handshake-specific definitions of bytes 48-1535.

TABLE 2 Handshake command block format Bytes Field 0-7 ProtocolSignature  8-15 Checksum 16-19 Version 20-23 Reserved 24-27 HandshakeOpcode 28-31 DEK management protocol Response Status 32-39 DEKmanagement protocol Endpoint ID 40-47 Key Controller Handle  48-303 (256bytes) Encryption Endpoint Class Name 304-511 (208 Bytes) Reserved 512-1023 (512 Bytes) Pseudo-random bytes 1024-1151 (128 Bytes) Reserved1152-1407 (256 Bytes) Pseudo-random bytes 1408-1535 (128 Bytes) Reserved

The following is a description of the various fields in the Handshakecommand block shown in Table 2 above:

-   -   DEK management protocol Endpoint ID—8 bytes—[Out] Returned by        the EE and for its internal use: the KCM 74 echoes (and thus        addresses) this Endpoint ID in subsequent Query, Associate, and        Disassociate commands for the Device Object. (An IL 76 creating        a protocol command would use this Endpoint ID to address the        EE.)    -   Endpoint Class Name—256 bytes—[In] Each class of EE has a name,        expressed, for example, as a null-terminated ASCII string.        Example endpoint class names include: “EHBA” for an HBA 56 with        hardware assist (i.e., an EHBA 156);    -   “DDM” for a data de-duplication module, which is typically an        external intermediate device; and “RRM” for a remote replication        module, which is typically an external intermediate device.    -   Pseudo-random bytes [512-1023]—512 bytes—[In/Out] “Handshake”        region #1 is filled with pseudo-random data by the KCM 74 and        sent down to the EE. The EE signals its presence in the storage        I/O stack 66 to the KCM 74 by, for example, reversing the order        of all 512 bytes in this region. This field is passed through        untouched by ILs 76, although if multiple Handshake response        command blocks are received from below, the IL 76 will choose        the field from the appropriate received Handshake response        command block to indicate affirmative or not, as appropriate.        This field may also be generated by an IL that creates a        command.    -   Pseudo-random bytes [1152-1407]—256 bytes—[In/Out] “Handshake        region #2 is similar to “handshake” region #1. The Encryption        Endpoint reverses all 256 bytes in this region before returning        a response to the command.    -   Reserved bytes [20-23, 304-511, 1024-1151, 1408-1535]—468        bytes—Undefined. Reserved for future use. Set to 0x00 by the KCM        74, ignored by the IL 76 and EE. These bytes are covered by the        checksum. An IL 76 should not overwrite them. They are included        in checksum to guard against false positive of a SCSI or other        I/O command being interpreted as a DEK management protocol        command.

The EE is expected to update the version field if the version supportedby EE is different than requested by KCM 74. The ILs 76 are alsoexpected to ensure version compatibility with the EE. If the EE supportsa lower version than required by the IL 76, IL 76 should fail theHandshake request.

Referring again to FIG. 6, in step 1016, KCM 74 sends a Query commanddown to the next IL 76 to determine if an encryption capability such ashardware assisted encryption is supported for a specified range on anLV. Each IL 76 between the KCM 74 and the EE responds to the Query basedon the encryption capabilities of the underlying devices.

An IL 76 broadcasts the Query command to all the underlying devices andaggregates the results of individual queries into one response to theKCM 74 (or an IL 76 above it). The response from an IL 76 should notlead to data corruption. For example, an IL managing a virtual volumespanning two underlying LUs should support hardware assisted encryptionon the virtual volume only if the paths to both the LUs have hardwareassist available.

For example, if the IL 76 is a virtualization module 80 that virtualizesa logical volume across two storage devices 40(a) and 40(b), thenvirtualization module 80 sends two copies of the Query command blockdown the storage I/O stack 66, one referencing storage device 40(a) asthe device object, and the other referencing storage device 40(b) as thedevice object. Generally, only if the virtualization module 80 receivesaffirmative Query response command blocks for both storage devices 40(a)and 40(b) will the virtualization module 80 respond back to the KCM 74with an affirmative response, however, this behavior may differ if aparticular form of virtualization is performed that allows otherwise.For example, in the case of a read-only LV mirrored onto two or moredistinct LUs, as long as one of the LUs is readable with encryption atthe level of an EHBA 156, the virtualizing IL may return an affirmativeresponse, even if a negative response is returned for one of the LUs.

As an alternate example, if the IL 76 is a multipathing module 82 havingpaths through multiple HBAs 56 to a given storage device 40, then themultipathing module 82 sends copies of the Query command block to allsuch HBAs down the storage I/O stack 66. If the multipathing module 82receives any affirmative Query response command blocks, thevirtualization module 80 respond back to the KCM 74 with an affirmativeresponse.

An EE looks for the Endpoint ID in the payload that matches its ID(i.e., the Endpoint ID that is sent up by the EE to the KCM 74 in theHandshake response), and returns affirmatively if it can perform itsencryption capabilities on the specified ranges for the device object.Otherwise the EE may return in the negative (e.g., if the EE does nothave a connection to the appropriate storage device 40, if the EE wasnot initialized, or if the EE is temporarily busy and the command shouldbe retried).

Included within the Query command is a LUN Map, which defines the areassubject to encryption. Each area is provided with reference to a LogicalBlock Address (LBA), which is an abstraction of the block addresses at agiven layer of logical abstraction. Returning to the example providedabove in which logical disk L1 88(a) is an encrypted LV 10 megabytes insize, blocks 1,536-10,239 and 11,264-20,479 of logical disk L1 88(a)would be listed as subject to encryption.

Some ILs 76 may remap the LUN map as appropriate. These ILs 76 arereferred to as “remapping” ILs 76. For example, a virtualization module80 is an example of a remapping IL 76, while a typical multipathingmodule 82 is not a remapping IL 76. Recall that, in the example, blocks0-10,239 of logical disk L1 88(a) are stored on D1 140(1), while blocks10,240-20,479 are stored on D4 140(4). Further suppose that theencrypted blocks stored on D1 140(1) begin at local block 1,000,000,while the encrypted blocks stored on D4 140(4), begin at local block2,097,152, but actually are spread out across 2 ranges:2,097,152-2,101,759 and 3,145,728-3,150,335. Therefore, in the Querycommand passed on to storage device D1 140(1), the LUN Map will indicateLBAs 1,000,000-1,008,703; and in the Query command passed on to storagedevice D4 140(4), the LUN Map will indicate LBAs 2,097,152-2,101,759 and3,145,728-3,150,335.

See Table 3, below, for an example layout of a Query command block.

TABLE 3 Query command block format Bytes Field 0-7 Protocol Signature 8-15 Checksum 16-19 Version 20-23 Reserved 24-27 Query Opcode 28-31 DEKmanagement protocol Response Status 32-39 DEK management protocolEndpoint ID 40-47 Key Controller Handle, Undefined 48-71 (24 bytes)Undefined 72-75 (4 bytes) LUN Map Count 76-83 (8 bytes) Starting LBAEntry [0] 84-91 (8 bytes) Number of Blocks [0] 92-99 (8 bytes) StartingLBA Entry [1] 100-107 (8 bytes) Number of Blocks [1]  108-1019 LBA RangeStructures [2] to [58] 1020-1027 (8 bytes) Starting LBA Entry [59]1028-1035 (8 bytes) Number of Blocks [59] 1036-1535 Reserved

The following is a description of the various fields in the Querycommand block shown in Table 3 above:

-   -   DEK management protocol Endpoint ID—8 bytes—[In] Returned by the        EE in the Handshake command response, echoed back by KCM 74,        thus addressing the Endpoint ID.    -   Undefined bytes [48-71]—24 bytes—[In/Out] Undefined, can be        anything. Included in checksum.    -   LUN Map Count—4 bytes—[In] Number of valid LUN Map entries being        queried. Must be at least one and not greater than the total        entries that can fit in the read buffer, (e.g., 60.) The IL 76        validates the map.    -   LUN Map Entry—16 to 960 bytes (up to 60 16-byte structures)—[In]        The LUN map is a list of LBA ranges on the LU. Each LUN Map        entry contains 2 sub-entries, each of which is, for example, a        64-bit integer: a starting LBA; and a number of blocks. Any        remapping IL 76 can adjust the starting LBA and/or number of        blocks as the request for association flows down the stack.    -   Reserved bytes [1036-1535]—Undefined and reserved for future        use.

Recall that, if step 1010 returns an affirmative response, executionproceeds with step 1050, while if step 1010 returns a negative response,execution may proceed with step 1070. In some embodiments, step 1050 mayalso be executed on its own, without first performing step 1010.

In step 1050, KCM 74 sends encryption metadata associated with theencrypted LV from the KCM 74 to the EE via ILs 76, the encryptionmetadata identifying an encryption key and one or more encrypted regionsof the LV. The encryption metadata may also identify other associateencryption information needed to perform the encryption algorithm, suchas, for example, an identification of the encryption algorithm. Thesending results in establishment of one or more shared associationsbetween the KCM 74 and the EE, the shared associations associating theencrypted LV with the encryption metadata for the encrypted LV. In oneembodiment, this step is accomplished using the DEK management protocolby sending a DEK Management Associate command.

The Associate command creates an association of (1) an Encryption KeyBlob, with (2) a LUN Map on (3) a Device Object, thereby effectivelyturning on encryption for the LU and LBA Range(s). The Key Blob is a setof encryption metadata, storing the key and all the other informationneeded to perform encryption/decryption that is stored on the keymanager, as described below. Although in the on-host case, the key blobis sent within the Associate command, in an off-host case, the key IDmay be sent within the Associate command instead of the key blob (or, insome embodiments, an encrypted version of the key blob, referred to as a“wrapped” key blob, may be sent). Multiple Key Blob/LUN Map Associationscan be made for a Device Object. Associate commands can be generated bythe KCM 74 and by ILs 76, although ILs 76 do not originate anassociation, but rather pass on one or more copies (with modificationsas necessary) of an Associate command received from above. In somecases, the association may also include Application information.

There are two forms of an Associate command:

-   -   New Association—creates a new association. In the case of a new        association, the Associate command block arrives at the EE or IL        76 with a Null “Association Handle” (see below). This tells the        EE/IL 76 that this association does not currently exist, that it        should be created and that an Association Handle reference        should be created and returned in the Associate response.    -   Refresh Association—the Refresh Association originates from the        KCM 74 and exists for the benefit of the ILs 76 or the EE. In        the case of a Refresh Association, the Associate command block        arrives at the EE or IL 76 carrying the Association Handle        created by the EE (or an IL 74) as part of a preceding initial        association.

An EE should respond as follows for the different Associationtypes/association handle values:

If the Association Handle is NULL—it means the KCM 74 or an IL 76 iscreating a new Association, so the EE should:

-   -   Validate the parameters as needed (see below).    -   Create the Association.    -   Return a Handle for the Association.    -   If the EE already has an association, provided there is no range        overlap, it should ignore the existing association and treat the        request as a new association.

If the Association Handle is not Null—it means the Association exists,so the EE should:

-   -   If the Associate carries the same LUN Map and Key Blob specified        in the original Associate, then return Success status.    -   Else—something is wrong, this should not happen—so respond        negatively by returning an Association Exists status.

Any Associate command (whether the first or a repeat) should be precededby a Query command—though the EE does not need to enforce this.

FIG. 7 shows an example layout of an Associate command block. Thefollowing is a description of the various fields in the Associatecommand block shown in FIG. 7:

-   -   DEK management protocol Response Status—4 bytes—[Out] possible        returns are Success, Invalid HW State, No Memory, Busy, Invalid        Range, Invalid Key, Association Exists, Association Overflow.    -   DEK management protocol Endpoint ID—8 bytes—[In] Echoed from the        EE's response to the initial Handshake command. Address of the        EE for the Associate. The EE passes on an Associate command that        does not contain the EE's Endpoint ID.    -   Association Handle—8 bytes—[In/Out]        -   [In] Zero—first time Association. An Association Handle is            returned by the EE or IL 76. The handle is an internal value            used by the EE or IL 76 for accessing an association. The            Association Handle is subsequently passed back down by the            KCM 74 to the EE in Update and Disassociate commands. An EE            assigns a unique association handle for each association            created. ILs 76 may need to replace the association handles            based on their internal device mappings, so that a single            handle is returned to the KCM 74. An IL 76 keeps track of            the handle(s) returned from below it and uses those handles            for passing down any subsequent Associate or Disassociate            command.        -   [In] Non-zero implies KCM 74 is attempting to refresh an            existing association. When dispatching it to the newly            discovered devices, the ILs 76 should zero out the            association handle and replace the handle with the new            handle on the way up to KCM 74.    -   Data Encryption Key Parameters—    -   The association handle is followed by offsets to various data        items 304:        -   Key Blob 304 (4 bytes) (offset shown as 302 in FIG. 7)        -   Key ID (4 bytes) (offset shown as 306 in FIG. 7)        -   Application Info (4 bytes) (offset shown as 308 in FIG. 7)        -   LUN Map (4 bytes) (offset shown as 310 in FIG. 7)    -   These offsets 302, 306, 308, 310 are followed by the following        variable length parameters:    -   Key Blob 304        -   Key Blob Length—4 bytes[In]—The number of bytes in the key            blob        -   Key Blob Type—1 byte [In]—This field indicates whether the            format of the key blob is “wrapped” (i.e., encrypted, as,            for example, it may be when being sent off-host to an            external intermediate device 42 or when being sent within a            highly-secure system) or “unwrapped” (i.e., unencrypted, as            for example, it may be when being sent to an EHBA 156 within            the host 36).        -   Key Data        -   Key Data Version—1 byte [In]—Versioning information for the            key data        -   Key Data Length—1 byte [In]—Length of the symmetric key        -   Key Algorithm—1 byte [In]—Algorithm        -   Key Mode—1 byte [In]—Algorithm Mode        -   Key Data—64 bytes [In]—Carries the key data of the length            “Key Data Length”. Extra bytes, if any are, zero.        -   Application info size—1 byte—[In] maximum accepted size of            the application information.    -   Key ID        -   Key ID Length—4 bytes [In]—Number of bytes in key ID        -   Key ID bytes—[In]—Key ID bytes    -   LUN Map Count—4 bytes [In]—Number of valid LUN Map entries being        reported. It should be at least one. Implementations can        restrict the number of LUN map entries supported.    -   LUN Map Array—16 to 960 bytes (up to 60 16-byte structures)—[In]        Specifies the LBA ranges on the Device Object to associate with        the Key Blob 304 or Key ID. Sub-fields include starting LBA and        a length or LBA-count. Unused map entries are set to zero.    -   Reserved bytes [variable-1535]—Unused and undefined

Upon successful completion of an Associate during step 1050, an EE isready to apply encryption/decryption to the encrypted regions of a LU asdefined in the LUN map, using the encryption metadata from the Key Bloband the application information. As long as the association remainsactive, subsequent read/write commands directed to these regions employdecryption/encryption using the encryption metadata. This operation isdepicted in step 1060.

The DEK management protocol may also employ Update and Disassociatecommands. An Update command tells the EE to update the association forthe Device Object with the Key Object and LUN map information in theprotocol command block. It provides an atomic way for an EE toeffectively delete and create an association in one step. It would beused, for example, to support resizing of an encrypted LU.

The Disassociate Command deletes the association that had been createdwith a previous Associate command for a Device Object. Subsequent readand write commands in the LBA range(s) covered for that association areno longer encrypted/decrypted by the EE. Disassociate is used when theEE can no longer perform its duties and a switch to encrypting using thecrypto kernel 84 is needed. Switching back happens through a newAssociate command. An example, looking back at FIG. 4, would be if EHBA1failed for some reason. D1 and D4 would still be reachable by EHBA3 andNHBA1, respectively, but the Crypto kernel 84 would have to be used sothe Disassociate would be sent on L1.

Both the Update (which, in some embodiments, is an Associate commandcontaining a valid non-null handle) and Disassociate commands include anAssociation Handle to identify the subject association.

In some embodiments, the invention may be applied to the sharing ofencryption metadata with an external encryption endpoint such as theexternal intermediate device 42. FIG. 8 depicts a method 1100 which maybe performed by an external intermediate device 42. At step 1110, keyidentification information (e.g., the key ID) is retrieved from a keycontroller module (e.g., KCM 74) on an external host (e.g., host 36). At1120, a key which is identified by the key identification information isobtained from an external key server (e.g., key manager server 32). Insome embodiments, this is done by sending the key ID from the externalintermediate device 42 to key manager server 32, which then responds bysending key blob to external intermediate device 42 securely across hostinterconnect 34. The key is sent indirectly in this manner becausesecurity restrictions typically prevent an encryption key from beingsent off-host. The key must be retrieved separately from the key managerserver 32 in order to enforce access limitations. In some cases, it maybe possible to instead send the key directly to the externalintermediate device 42 if it is embedded within an encrypted “wrapped”key blob. It should be understood that, in some embodiments, step 1110may be performed using a command similar to the Associate commanddepicted in FIG. 7, however, the key blob 304 would be either omittedfrom the command block or sent in wrapped format.

At step 1130, the key is used for encrypting/decrypting data, and, atstep 1140, the encrypted/decrypted data is processed appropriately.

Two specific examples are shown in FIG. 8. In one case, the externalintermediate device 42 provides a remote replication function toreplicate storage volumes on a remote storage device 46. To makeefficient use of long-distance communications bandwidth, it is desirableto compress the data prior to transmitting it. However, encrypted databenefits very little from compression, because encryption generallyremoves the large redundancy that is present in cleartext data which isexploited by compression techniques. Thus, the external intermediatedevice 42 in this case first reads and decrypts data of a local storagedevice 40 which is being replicated (step 1130), compresses thedecrypted data (step 1142), encrypts the compressed data (1144) and thensends the encrypted, compressed data across the tunnel 44 to the remotestorage device 46 (1146). In the event of needing to retrieve data fromthe remote storage device 46 for local use, the external intermediatedevice 42 performs these operations in reverse.

Steps 1152-1156 depict a set of similar operations for so-called “datade-duplication”, a function that also exploits redundancy and thus isbetter performed on cleartext rather than encrypted data.

It should be understood that in either the remote replication case orthe data de-duplication case the external intermediate device 42 mayalternatively receive the encrypted data directly from an HBA 56 (e.g.,via storage interconnect 38) rather than by reading it from a storagedevice 40.

While various embodiments of the invention have been particularly shownand described, it will be understood by those skilled in the art thatvarious changes in form and details may be made therein withoutdeparting from the spirit and scope of the invention as defined by theappended claims.

It should be understood that although various embodiments have beendescribed as being methods, software embodying these methods is alsoincluded. Thus, one embodiment includes a tangible computer-readablemedium (such as, for example, a hard disk, a floppy disk, an opticaldisk, computer memory, flash memory, etc.) programmed with instructions,which, when performed by a computer or a set of computers, cause one ormore of the methods described in various embodiments to be performed.Another embodiment includes a computer which is programmed to performone or more of the methods described in various embodiments.

Furthermore, it should be understood that all embodiments which havebeen described may be combined in all possible combinations with eachother, except to the extent that such combinations have been explicitlyexcluded.

Finally, nothing in this Specification shall be construed as anadmission of any sort. Even if a technique, method, apparatus, or otherconcept is specifically labeled as “prior art” or as “conventional,”Applicants make no admission that such technique, method, apparatus, orother concept is actually prior art under 35 U.S.C. §102, suchdetermination being a legal determination that depends upon manyfactors, not all of which are known to Applicants at this time.

What is claimed is:
 1. A method, performed in a device, the methodcomprising: receiving key identification information from a keycontroller module on an external host; obtaining a key identified by thekey identification information from an external key server; decryptingencrypted data from an encrypted storage system using the key; andprocessing the decrypted data; wherein receiving key identificationinformation from the key controller module includes: receiving anassociate command from the key controller module on the external host,the associate command including: an opcode field set to a value thatindicates that the establishment of an association is being attempted;the key identification information, the key identification informationincluding a key identifier, the key identifier serving to identify thekey without directly revealing the key; and a set of fields, the set offields identifying a set of data ranges subject to encryption; andsending an associate response to the key controller module on theexternal host in response to the associate command, the associateresponse including: a response field set to a value that indicateswhether the association has been successfully established; and a handlefield set to a handle value identifying the established association. 2.The method of claim 1 wherein processing includes: compressing thedecrypted data; re-encrypting the compressed data; and sending there-encrypted data across a wide area network for remote duplication. 3.The method of claim 1 wherein processing includes: de-duplicating thedecrypted data; re-encrypting the de-duplicated data; and storing there-encrypted data in the encrypted storage system.
 4. The method ofclaim 1, wherein the receiving key identification information from thekey controller module on the external host is performed only uponsuccessfully indicating to the key controller module on the externalhost that the device is able to cryptographically process data of a datastorage operation.
 5. The method of claim 4, wherein indicatingincludes: receiving a query command from the key controller module onthe external host, the query command including another set of fields,the other set of fields identifying the set of data ranges subject toencryption; and sending a query response to the key controller module onthe external host in response to the query command, the query responsehaving another response field set to a value that indicates whether allranges of the set of data ranges can be cryptographically processed bythe device.
 6. The method of claim 1, wherein obtaining the keyidentified by the key identification information from the external keyserver includes: sending the key identifier to the external key server;authenticating to the external key server; and receiving a key blob fromthe external key server, the key blob being encrypted and including: thekey identified by the key identification information to be used inperforming cryptographic data storage operations on the device; and anindication of an encryption algorithm to be used in performingcryptographic data storage operations on the device.
 7. A method,performed in a device, the method comprising: receiving keyidentification information from a key controller module on an externalhost; obtaining a key identified by the key identification informationfrom an external key server; decrypting encrypted data from an encryptedstorage system using the key; and processing the decrypted data,wherein: the receiving key identification information from the keycontroller module on the external host is performed only uponsuccessfully indicating to the key controller module on the externalhost that the device is able to cryptographically process data of a datastorage operation; and indicating includes: receiving a query commandfrom the key controller module on the external host, the query commandincluding: an opcode field set to a value that indicates that a query isto be performed; and a set of fields, the set of fields identifying aset of data ranges subject to encryption; and sending a query responseto the key controller module on the external host in response to thequery command, the query response having a response field set to a valuethat indicates whether all ranges of the set of data ranges can becryptographically processed by the device.
 8. The method of claim 7,wherein indicating further includes, prior to receiving the querycommand: receiving a handshake command from the key controller module onthe external host, the handshake command including: an opcode field setto a value that indicates that a handshake is to be performed; and ahandshake field containing a set of pseudo-random data bytes; andsending a handshake response to the key controller module on theexternal host in response to the handshake command, the handshakeresponse including: a response field set to a value that indicates thatthe device exists; and a handshake response field, the handshakeresponse field containing the set of pseudo-random data bytes with aspecific modification.
 9. The method of claim 7 wherein receiving keyidentification information from the key controller module includes,after sending the query response: receiving an associate command fromthe key controller module on the external host, the associate commandincluding: an opcode field set to a value that indicates that theestablishment of an association is being attempted; and a keyidentifier; and sending an associate response to the key controllermodule on the external host in response to the associate command, theassociate response including: a response field set to a value thatindicates whether the association has been successfully established; anda handle field set to a handle value identifying the establishedassociation.
 10. The method of claim 9, wherein the associate commandfurther includes the set of fields identifying the set of data rangessubject to encryption.
 11. The method of claim 9, wherein obtaining thekey identified by the key identification information from the externalkey server includes: sending the key identifier to the external keyserver; authenticating to the external key server; and receiving a keyblob from the external key server, the key blob being encrypted andincluding: the key identified by the key identification information tobe used in performing cryptographic data storage operations on thedevice; and an indication of an encryption algorithm to be used inperforming cryptographic data storage operations on the device.
 12. Acomputer program product comprising a non-transitory computer-readablestorage medium storing a set of instructions, which, when performed by acomputing device, cause the computing device to perform the operationsof: receiving key identification information from a key controllermodule on an external host; obtaining a key identified by the keyidentification information from an external key server; decryptingencrypted data from an encrypted storage system using the key; andprocessing the decrypted data; wherein receiving key identificationinformation from the key controller module includes: receiving anassociate command from the key controller module on the external host,the associate command including: an opcode field set to a value thatindicates that the establishment of an association is being attempted;the key identification information, the key identification informationincluding a key identifier, the key identifier serving to identify thekey without directly revealing the key; and a set of fields, the set offields identifying a set of data ranges subject to encryption; andsending an associate response to the key controller module on theexternal host in response to the associate command, the associateresponse including: a response field set to a value that indicateswhether the association has been successfully established; and a handlefield set to a handle value identifying the established association. 13.The computer program product of claim 12 wherein the instructions forprocessing include instructions, which when performed by the computingdevice, cause the computing device to perform the operations of:compressing the decrypted data; re-encrypting the compressed data; andsending the re-encrypted data across a wide area network for remoteduplication.
 14. The computer program product of claim 12 wherein theinstructions for processing include instructions, which when performedby the computing device, cause the computing device to perform theoperations of: de-duplicating the decrypted data; re-encrypting thede-duplicated data; and storing the re-encrypted data in the encryptedstorage system.
 15. The computer program product of claim 12, whereinthe instructions for receiving key identification information from thekey controller module on the external host are performed by thecomputing device only upon the computing device successfully indicatingto the key controller module on the external host that the computingdevice is able to cryptographically process data of a data storageoperation.
 16. A computer program product comprising a non-transitorycomputer-readable storage medium storing a set of instructions, which,when performed by a computing device, cause the computing device toperform the operations of: receiving key identification information froma key controller module on an external host; obtaining a key identifiedby the key identification information from an external key server;decrypting encrypted data from an encrypted storage system using thekey; and processing the decrypted data, wherein; the instructions forreceiving key identification information from the key controller moduleon the external host are performed by the computing device only upon thecomputing device successfully indicating to the key controller module onthe external host that the computing device is able to cryptographicallyprocess data of a data storage operation; and indicating includes:receiving, at the computing device, a query command from the keycontroller module on the external host, the query command including: anopcode field set to a value that indicates that a query is to beperformed; and a set of fields, the set of fields identifying a set ofdata ranges subject to encryption; and sending a query response from thecomputing device to the key controller module on the external host inresponse to the query command, the query response having a responsefield set to a value that indicates whether all ranges of the set ofdata ranges can be cryptographically processed by the computing device.17. The computer program product of claim 16, wherein indicating furtherincludes, prior to receiving the query command: receiving, at thecomputing device, a handshake command from the key controller module onthe external host, the handshake command including: an opcode field setto a value that indicates that a handshake is to be performed; and ahandshake field containing a set of pseudo-random data bytes; andsending a handshake response from the computing device to the keycontroller module on the external host in response to the handshakecommand, the handshake response including: a response field set to avalue that indicates that the computing device exists; and a handshakeresponse field, the handshake response field containing the set ofpseudo-random data bytes with a specific modification.
 18. The computerprogram product of claim 16 wherein the instructions for receiving keyidentification information from the key controller module includeinstructions, which when performed by the computing device, cause thecomputing device to, after sending the query response, perform theoperations of: receiving an associate command from the key controllermodule on the external host, the associate command including: an opcodefield set to a value that indicates that the establishment of anassociation is being attempted; and a key identifier; and sending anassociate response to the key controller module on the external host inresponse to the associate command, the associate response including: aresponse field set to a value that indicates whether the association hasbeen successfully established; and a handle field set to a handle valueidentifying the established association.
 19. The computer programproduct of claim 18, wherein the associate command further includes theset of fields identifying the set of data ranges subject to encryption.20. The computer program product of claim 18, wherein the instructionsfor obtaining the key identified by the key identification informationfrom the external key server include instructions, which when performedby the computing device, cause the computing device to perform theoperations of: sending the key identifier to the external key server;authenticating to the external key server; and receiving a key blob fromthe external key server, the key blob being encrypted and including: thekey identified by the key identification information to be used inperforming cryptographic data storage operations on the computingdevice; and an indication of an encryption algorithm to be used inperforming cryptographic data storage operations on the computingdevice.
 21. A system comprising: a set of storage devices providing datastorage; a host computer configured to process data storage commandsaimed at the set of storage devices, the host computer being connectedto storage devices of the set of storage devices via a set of storagepaths across a network, the host computer including a storage I/O stackhaving a key controller module, the key controller module beingconfigured to manage data encryption for data storage operationsprocessed through the storage I/O stack; a key management serverexternal to the host computer configured to manage keys for differentportions of the data storage; an intermediate device external to thehost computer and external to the key management server, theintermediate device being configured to: receive key identificationinformation from the key controller module on the host computer; obtain,from the key management server, a key identified by the keyidentification information; decrypt encrypted data from the set ofstorage devices using the key; and process the decrypted data; whereinreceiving key identification information from the key controller moduleincludes: receiving an associate command from the key controller moduleon the host computer, the associate command including: an opcode fieldset to a value that indicates that the establishment of an associationis being attempted; the key identification information, the keyidentification information including a key identifier, the keyidentifier serving to identify the key without directly revealing thekey; and a set of fields, the set of fields identifying a set of dataranges subject to encryption; and sending an associate response to thekey controller module on the host computer in response to the associatecommand, the associate response including: a response field set to avalue that indicates whether the association has been successfullyestablished; and a handle field set to a handle value identifying theestablished association.